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Introduction to the Seminar Guide 



This Seminar guide was created by the National Center for the Study of Adult 
Learning and Literacy (NCSALL). The guide is part of NCSALL’s effort to 
help connect research and practice in the field of adult education and family 
literacy. 

This Seminar Guide — Establishing an Evidence-based Adult Education 
System — is one of many teaching and training tools that NCSALL has 
developed. It is meant to accompany NCSALL’s Occasional Paper entitled 
“Establishing an Evidence-based Adult Education System” (available at 
http://ncsall.gse.harvard.edu/research/op_comings3.pdf). 

The purpose of the three-hour seminar is to help practitioners, program 
directors, state staff, and other policymakers read and discuss NCSALL’s 
proposal (contained in the paper) for creating an evidence-based system for 
our field and then to understand and judge the relevance of such a system to 
their work in adult basic education. 

The Seminar Guide is divided into the following parts: 

• Preparing to Facilitate the Seminar 

• Session Preparation: What To Have Ready 

• Session Guide for Facilitating the Seminar 

• Handouts, Readings, Recruitment Materials, and Background Information 
for the Facilitator 

We welcome your comments and advice about this Seminar and about 
NCSALL. For more information about NCSALL’s work, contact: 

Cristine Smith, NCSALL Deputy Director 

NCSALL at World Education 

44 Farnsworth Street 

Boston, MA 02210-1211 

(617) 482-9485 

ncsall@worlded.org 



OR VISIT OUR WEB SITE: 

http://ncsall.gse.harvard.edu 
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Information About NCSALL 



The Mission of NCSALL. NCSALL’ s purpose is to improve practice in 
educational programs that serve adults with limited literacy and English 
language skills, and those without a high school diploma. NCSALL is 
meeting this purpose through basic and applied research, dissemination of 
research findings, and leadership within the field of adult learning and 
literacy. 

NCSALL is a collaborative effort among the Harvard Graduate School of 
Education, World Education, The Center for Literacy Studies at The 
University of Tennessee, Rutgers University, and Portland State University. 
NCSALL is funded by the U.S. Department of Education through its Office of 
Educational Research and Improvement (OERI). 

NCSALL’s Research Projects. The goal of NCSALL’s research is to 
provide information that is used to improve practice in programs that offer 
adult basic education (ABE), English for Speakers of Other Languages 
(ESOL), and adult secondary education services. In pursuit of this goal, 
NCSALL has undertaken research projects in four areas: (1) learner 
motivation, (2) instructional practice and the teaching/leaming interaction, 

(3) staff development, and (4) assessment. 

Dissemination Initiative. NCSALL’s dissemination initiative focuses on 
ensuring that practitioners, administrators, policymakers, and scholars of adult 
education can access, understand, judge, and use research findings. NCSALL 
publishes Focus on Basics, a quarterly magazine for practitioners; Focus on 
Policy, a twice-yearly magazine for policymakers; Review of Adult Learning 
and Literacy, an annual scholarly review of major issues, current research, and 
best practices; and NCSALL Reports and Occasioned Papers, periodic 
publications of research reports and articles. NCSALL also sponsors the 
Connecting Practice, Policy, and Research Initiative, designed to help 
practitioners and policymakers apply findings from research in their 
instructional settings and programs. Lor more information about NCSALL’s 
publications and activities, please visit our web site at: 



http:// ncsall.gse.harvard.edu 
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Preparing to Facilitate This Seminar 

What’s in This Guide? 

This Guide includes all of the information and materials you will need to 
conduct the Seminar, including a step-by-step guide of the activities included 
in the session, the approximate time the activity should take, and notes and 
other ideas for conducting the activities. At the end of the session guide, you 
will find the handouts and reading for the Seminar, ready for photocopying. 

Who Should Participate in This Seminar? 

This Seminar is designed for any practitioner who works in adult education: 
teachers, program directors, state staff, counselors, or others. The design of 
the Seminar should be appropriate for practitioners with some experience in 
the field. 

The Seminar is designed for approximately 8 to 15 participants. We don’t 
recommend running the Seminar with fewer than five or more than 15 people, 
as discussions are harder to facilitate with too few or too many participants. 

How Long Is the Seminar? 

The Seminar is designed to be conducted over one 3-hour period. However, it 
could be divided into two 1 .5 hour sessions on different days, if that is easier 
to implement within a program. If participants want follow-up after the last 
session, they can set that up during the last session. 

How Do I Recruit Participants? 

We predict that programs, coalitions, or state departments of education will 
use this Seminar in place of a regularly-scheduled meeting of participants: a 
statewide meeting of program directors, a program staff meeting, a coalition 
meeting. 

However, if you would like to recruit a group of practitioners from different 
areas or programs to discuss the “Establishing an Evidence-based Adult 
Education System” paper, there’s a sample recruitment flyer in Appendix C 
that provides information about this NCSALL Seminar. The flyer presents an 
overview of the topic and a brief description about what will be covered in the 
Seminar. It also includes a registration form for practitioners to mail back to 
you. 
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How Do I Prepare for the Seminar? 

When you feel that you have solid commitments from the number of 
participants you are seeking, contact them to confirm their participation in the 
Seminar. At this time, reconfirm the time, date, and location of the 
Seminar with the participants. 

To prepare, we suggest you send the following materials to the participants 
before the Seminar: 

• a copy of the Establishing an Evidence-based Adult Education System 
paper can be downloaded and printed from NCSALL’ s Web site 
( http://ncsall.gse.harvard.edu/research/op comings3.pdf ) and photocopied, 
or ordered in bound, printed version from NCSALL by e-mailing us at 
ncsall@worlded.org or by phone at (617) 482-9485); and 

• a brief cover letter with information about the Seminar to send to 
participants before the Seminar. (You will find the sample cover letter in 
Appendix C of this guide). 

Participants should receive this packet at least 10 days before the first 
meeting of the Seminar. You, as the facilitator, will also want to read the 
paper so that you are ready for the first session and for questions that 
participants may have. 

Be sure that you arrive early for the Seminar in order to set up the chairs in a 
circle; arrange your newsprint; and organize handouts, pens, tapes, etc. 
Consider having some sort of refreshment, such as juice, coffee, or water. 
Check out where the restrooms are so you can tell participants where they are 
located, and make sure the heating or cooling in the room is appropriate. 

You may also want to have (or start during the meeting) a list of participants 
and their phone numbers or e-mail addresses so participants can contact one 
another after the Seminar. Also, bring two or three extra copies of the paper. 
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What Kind of Support Can I Expect? 

For advice on facilitating the Seminar, if you want to talk the Seminar over 
beforehand with someone from NCSALL, you should feel free to contact us. 
We can be reached at: 



Cristine Smith, NCSALL Deputy 
Director 

NCSALL at World Education 
44 Farnsworth Street 
Boston, MA 02210-1211 
(617) 482-9485 
ncsall@worlded.org 



Beth Bingman, NCSALL Coordinator 
Center for Literacy Studies 
The University of Tennessee 
600 Henley Street, Suite 312 
Knoxville, TN 37996-2135 
(865) 974-6618 
bingman@utk.edu 



What If I Want to Change the Activities in the Guide? 

Feel free. The Guide is intended to help first-time facilitators who want a lot 
of guidance. It is not intended to be a script that must be followed. If you 
have a different idea for how to do an activity, you should use it. To help you, 
we have offered alternative ideas for several of the activities. 

What Should the Outcomes Be of the Seminar? 

First and foremost, we hope that individual practitioners learn about evidence- 
based practice and come away with ideas for how it is relevant to their own 
classrooms and programs. Secondly, we hope that the group creates a plan for 
further work. The group might decide to: 

• have a follow-up meeting to discuss how the concept of 
evidence-based practice can be applied in their programs or 
state, 

• stay in touch and share ideas via e-mail, 

• produce a summary of what they’ve learned - something 
practical - that can be shared with other practitioners or 
policymakers, 

• develop a list of policy changes at the program or state level 
that would support the establishment of an evidence-based 
adult education system, or 

• decide on some action to take in their program or state to learn 
more about or promote evidence-based practice. 
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Session Preparation 



Newsprint (Prepare ahead of time) 

Objectives and Agenda (p. 8) 

Evidence-based practice is... (p. 9) 

Elements of a Program Model (p. 10) 

Goal: Using an evidence-based model for 
providing services to students (p. 11) 

Some Next Steps (p. 11) 



Handouts (Make copies for each participant) 

Handout 1 : Components of an EBAE System 
(Appendix A) 

Handout 2: Elements of a Program Model 
(Appendix A) 



Reading (Have two or three extra copies 



available for participants who forget to bring 
theirs) 



Establishing an Evidence-based Adult 
Education System (NCSALL Occasional Paper) 



Materials 

Newsprint easel 
Markers, pens, tape 
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Session Guide 

Objectives: 



Participants will: 

• consider the definition of evidence-based practice and its 
relevance to their classrooms, program, state, and the field of 
adult education; 

• discuss the components of an evidence-based adult education 
system, and the usefulness of baseline program models for their 
own work in adult education; 

• consider next steps for supporting or being involved in the 
development of an evidence-based adult education system, at 
all levels. 

Time: 3 hours 



1 0 minutes 


1 . 


Welcome and Introductions 


5 minutes 


2. 


Objectives and Agenda 


25 minutes 


3. 


The Definition of Evidence-based Practice 


25 minutes 


4. 


Components of an Evidence-based Adult 
Education (EBAE) System 


15 minutes 




Break 


65 minutes 


5. 


Elements of Baseline Program Models 


30 minutes 


6. 


Next Steps for Supporting an EBAE System 


5 minutes 


7. 


Evaluation of the Seminar 
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Steps 

1. Welcome and Introductions (io minutes) 



• Welcome participants to the Seminar. Introduce yourself and state 
your role as facilitator. Explain how you came to facilitate this 
Seminar and who is sponsoring it. 









Ask participants to introduce themselves briefly (name, 
program, role). You could also ask them to very briefly add 
something more personal to their introduction, such as 
describing one aspect of their journey to this Seminar. 

Make sure that participants know where bathrooms are 
located, when the session will end, when the break will be, and 
any other “housekeeping” information. 



Note to Facilitator 

Since time is very tight, 
it’s important to move 
participants along 
gently but firmly if they 
are exceeding their 
time limit for 
introductions. 



2. Objectives and Agenda 



(5 minutes) 



• Post the Objectives and Agenda newsprint and review with 
participants. 



Objectives 

• Consider the definition of evidence-based practice 

• Discuss the components of an evidence-based adult 
education system and the usefulness of baseline program 
models 

• Consider next steps for supporting or being involved in the 
development of an evidence-based adult education 
system 



Agenda 

1 . Introduction, objectives, and agenda (Done!) 

2. Discussion of Definition of evidence-based practice 

3. Discussion of Components and baseline models 

4. Planning next steps 
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3. The Definition of Evidence-Based Practice (25 minutes) 



• Introduce this session by referring to pages 1-3 in the paper 
0 Establishing an Evidence-based Adult Education System). The 
rationale for discussing the definition provided in the paper is that the 
term is relatively new to many in our field yet evidence-based practice 
(EBP) is being heavily promoted and programs may be required to 
demonstrate how their services for learners are based on evidence. 

The definition used in this paper is that used by the Assistant Secretary 
of Education, Grover Whitehurst. 



• Post newsprint with the definition of EBP on it and a question for 
discussion: 



Note to Facilitator 

For more information 
about this definition, 
please see the 
Background Materials 
for Facilitator in 
Appendix D. 



Evidence-based practice is: 

...the integration of professional wisdom with the best available 
empirical evidence in making decisions about how to deliver 
instruction. 



Question : 

What does this definition mean to you, and, if adopted, what 
are the implications for your program? 




• Ask participants to discuss the question (either in whole group if 
less than 5 participants, or in two or more small groups if group is 
more than 5) for 20 minutes. No need to write on newsprint; this is 
just a free-flowing discussion. 



4. Components of an EBAE system (25 minutes) 



• Introduce this session by referring to page 3 and page 12 in the paper 
(Establishing an Evidence-based Adult Education System). Building 
the system requires these three components and activities. It also 
requires program models for serving particular populations of learners 
(beginning level ESOL learners, GED learners, etc.). The next activity 
will help us discuss the components and the elements of a model. 

1 Distribute Handout #1 (found in Appendix A). Ask participants to 
look at the diagram and discuss these two questions (20 minutes): 
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? What questions do you have about these components? 

? What strikes you most about the role of research in developing 
the system? 



Break (15 minutes) 



5. Elements of Baseline Program Models (65 minutes) 



Distribute Handout #2 (found in Appendix A). Ask participants to 
break into small groups (or, if less than 5 participants, stay in the 
whole group), and take 20 minutes to discuss the questions written on 
newsprint. Give each group a copy of the same newsprint (with the 
questions), and ask one of the participants of the small group to make 
notes on the bottom of their newsprint to share in a report-back with 
the larger group afterwards. 



Elements of a Program Model 

In your small group, discuss: 

• What is the relevance of a model, containing principles for 
each of these elements, to your program(s)? 



• Models for which populations of students would be most 
helpful to you? 







Reconvene the whole group and ask each group to post their 
newsprints. Allow everyone time to read each others’ newsprints. 
Ask a representative from each group to answer any questions 
from the other group about their discussion (15 minutes). 

Facilitate a force field analysis (30 minutes) among the whole 
group, focused on the supports and barriers to using an evidence- 
based program model. Post the following newsprint: 



Note to Facilitator 

A force field analysis 
is a discussion and 
strategizing 
technique where a 
group of people 
brainstorm hindering 
forces, supporting 
forces, and then 
strategize how to 
reduce hindrances 
and increase 
supports. 
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Ask the group to brainstorm what would hinder them, their 
program, or organization in adopting an evidence-based model. Write 
their comments on the right side of the newsprint (under the “minus” 
sign). 

Ask the group to brainstorm what would support them, their 
program, or organization to adopt an evidence-based model. Write 
their comments on the left side of the newsprint (under the “plus” 
sign). 

Ask the group to brainstorm a few strategies for reducing the 
hindering factors and increasing the supportive factors so that a 
program model would be most helpful and most easily adopted. 



6. Next Steps for Supporting an EBAE System (30 minutes) 



• Introduce this session by referring to pages 16 and 17 in the EBAE 
paper. The paper has identified a series of next steps for establishing 
an EBAE system. This discussion will refer to these steps and help 
you think about what your group or organization wants to do. 



Some Next Steps 

What are the next steps your organization, program, or group 
needs to do to move toward an evidence-based adult education 
system? 
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Write up the next steps on the newsprint as the participants 
mention them. After 10 minutes of brainstorming, ask 
participants to silently look at the options and decide on two 
that they think are priorities. Then, ask participants to come 
forward and write a “1” and a “2” next to the two options 
they think are most important to do next. 



Another Idea 



Ask participants to 
brainstorm ways in 
which their organization 
could integrate research 
into practice and policy. 



• Tally the results and ask the group how it will move 
forward in implementing these steps. 



7. Evaluation of the Seminar 



(5 minutes) 



• Ask participants to evaluate the usefulness of this Seminar in terms 
of each objectives (refer to the “Objectives” newsprint). How 
could these activities be improved? 
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Appendix A 

Handouts 
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Handout #1: Components of an EBAE System 



Basic and 




Program Model 




Practitioner 


Applied 




Evaluations 




Knowledge 


Research 


► 




► 


Improves 


Builds Models 




Tests Models 




Implementation 
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Handout #2: Elements of a Program Model 



A program model describes what adult students, teachers, counselors, 
administrators, volunteers, and program partners (such as businesses, unions, 
health centers and training programs) should do to provide effective 
instruction and support services. 




Entrance into a program includes 

• Recruitment: the way in which 
programs describe and publicize 
their services to attract students 
who are making an informed 
choice to begin a course of study, 
and 

• the intake and orientation process: 
the way in which programs assess 
the needs and goals of students and 
prepare them to be successful in 
the program. 



Participation in a program 

includes 

• Instruction: the way in which 
programs support the learning of 
students so that they can achieve 
their goals, and 

• Support services: the way in 
which programs help students 
participate, persist, and engage 
in learning. 




Reengagement in learning is the way 

in which programs help students 
resume learning after they have left 
the program or begin postsecondary 
education or training after completion 
of program services. 




A baseline program model would establish principles for program services. 
A principle describes a guiding assumption about how best to organize the 
activities of the three elements. These principles would be derived from 
empirical evidence and practitioner knowledge. 
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Appendix B 

Reading 



Establishing an Evidence-Based Adult Education System 

by John Comings 

(To be read by participants BEFORE the session.) 

Download free from: http://ncsall.gse.harvard.edu/research/op comings3.pdf 
and reproduce. 

Or go to http://ncsall.gse.harvard.edu , click on “Publications,” then 
“Occasional Papers - By Title,” then on “Establishing an Evidence-based 
Adult Education System.” 

OR to order bound, printed versions from NCSALL, e-mail us at 
ncsall@worlded.org or call us at (617) 482-9485). The cost for mailing 
printed copies is $5.00 each. 
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Appendix C 

Cover Letter and 
Recruitment Flyer 
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Cover Letter 

(Send to participants with the Reading about 10 days prior to the Seminar.) 



DATE 



Dear Colleague: 

I am delighted that you are interested in participating in the 3-hour Seminar on 
Establishing an Evidence-based Adult Education System. 

The Seminar will be held on (DAY, DATE AND TIME) at (LOCATION). Attached you 
will find the reading for the Seminar. Please read this 20-page paper before the 
Seminar; our discussion during the Seminar will be based on this paper. There will not 
be time during the Seminar to read the paper, so in consideration of your fellow 
participants and for the sake of a rich discussion, please read the paper beforehand. 

If you have any questions about the Seminar or the reading, please contact me at 
(PHONE AND E-MAIL). 

(INFO ABOUT PARKING) 

(MAP, IF NECESSARY) 

I will provide refreshments during the Seminar. I look forward to seeing you there and to 
a productive discussion about this important topics. 

Sincerely, 



YOUR NAME 
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Recruitment Flyer 




Are you interested in discussing evidence-based 
systems for adult education? 

The 

( name of your agency or program) 

is sponsoring a Seminar to engage practitioners, program directors, state 
staff, and other policymakers in reading and then discussing the 
proposal by the National Center for the Study of Adult Learning and 
Literacy (NCSALL) for creating an evidence-based system for our field 
and then to understand and judge the relevance of such a system to their 
work in adult basic education. 

The resource book for this Seminar is NCSALL’ s Occasional Paper 
entitled “Establishing an Evidence-based Adult Education System,” 
written by NCSALL Director John Comings. As part of this Seminar, 
you will: 

• Learn about a model for an evidence-based system for adult 
education. 

• Discuss the components of an evidence-based adult education 
system, and the usefulness of baseline program models for your 
work in adult education. 

• Consider strategies for supporting or being involved in the 
development of an evidence-based adult education system. 

• Provide feedback and practical suggestions to NCSALL. 

The Seminar will consist of one session of approximately three hours. 
You will be asked to read “Establishing an Evidence-based Adult 
Education System” before the session for discussion during the Seminar. 
There is space for up to 15 participants. 

This is a staff development activity. CEUs or PDPs may be available. 
The date and location of the Seminar session are: 



If you would like to join the Seminar, please complete this form and 
send it to: [Facilitator’s name], NCSALL Practitioner Learner, 
[Facilitator’s address] by [deadline], I can also be reached at 
[Facilitator’s phone number] and [Facilitator’s email]. 



Name: Position: 

Home address: 

Home phone: Work phone: E-mail: 
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Appendix D: 

Background Materials 
for Facilitator 
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Why discuss evidence-based practice in education? 

Increasing emphasis in all of education for instruction and services to be based 
on evidence; i.e., practice should be based on evidence, not on intuition, fads, 
or trial and error. 

What counts as evidence-based practice? 

Grover (Russ) Whitehurst (Asst. Secretary of Education, head of the Institute 
of Education Sciences) has said that evidence-based education is: 

The integration of professional wisdom with the best available empirical 
evidence in making decisions about how to deliver instruction.” 

What is professional wisdom? 

The judgment that individuals acquire through experience. Professional 
wisdom includes expert opinion and practitioner knowledge. 

What is empirical evidence? 

Scientifically based research from fields such as psychology, sociology, 
economics, and neuroscience, and especially from research in educational 
settings. 

Objective measures of performance used to compare, evaluate, and monitor 
progress. 

Why are both professional wisdom and empirical evidence necessary for 
evidence-based practice? 

Without professional wisdom, education cannot : 

■ adapt to local circumstances. 

■ operate intelligently in the many areas in which research 
evidence is absent or incomplete. 

Without empirical evidence, education cannot : 

■ resolve competing approaches. 

■ generate cumulative knowledge. 

■ avoid fad, fancy, and personal bias. 

Distinction between evidence-based practice and scientifically based 
research 

Evidence-based practice : It’s about what should drive practice (using 
empirical evidence and professional wisdom to make decisions). 

Scientifically based research : It’s about what type of research should generate 
the empirical evidence (research that, according to DOE, meets particular 
criteria: experimental design, peer-refereed journal, sample size and selection, 
etc.). 
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Empirical evidence 

Research produces knowledge that can be used to design models of program 
service. 

Evaluation tests models of program service to see if they work or to see which 
of two or more models works best. 
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